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ABSTRACT 



The psychophysical technique of magnitude estimatxon 
was used to develop a ratio ^pale of subjective estimations of adults 
learning in. various adult education actd.vities. * A rank order of 26 
learning activities and -th*e magnitude estinfations in "uiiits of 
^learning" that^are expex:ted to occur in each activity were obtained 
from 1U6 adult education graduate and undergrarduate studentSr ' 165 
participants in' a. "school board adult education evening program and 
19i participants in credit courses off^re(J by a community college. 
Arithmetic and geometric means were computed and plotted for each of . 
the 26 learning activities,. Analysis of the result^s indicates that 
the magnitudes do" constitute a ratio scale. Analysis of covariance 
revealed, age and years of schooling completed to be significant ^ 
\sources of variance on seyefal- scale i^tems and sex was found to have 
a significant effect on one item. Test re-test reliability 
coefficients wi^h a one month interval between tests, ajid a • / 

cross-modal matching technique with a one week intq|nral between tests 
indicated ^the reliability of the Insttument. The . methodology of the 
study and the • implications of the* results for adult education ^ ^ 
research are discussed. A 14- item bibliqgraphy and a 6opy of the' 
instrument are included. (Author/MS) 
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ABSTRACT • 

The psychophysical technique of magnitude estimation was used 
to develop -a ratio ^cale of sub jective "estimations of adult learning in 
various adult • education activities. Previous research using magnitude 
estimation procedures has. revealed that it is possible to develop ratio 
scales to quantify certain social norms, opinions and^attitudes where 
judgements indicate how many times greater one situation is than 

■ — / ' . \ ' ' - ■ 

another. 

A geometric mean rank order of subjective estimates of learning in 
twenty-six adult education activities j and * the magnitudes of the estimates 
were obtained from a group of 191 coram'unity college participants, 147 
members of an adult education department and 165 p|irticipants. in a school 
district adult education program. Test re-test reliability coefficieats - 

with a on^ month interval between tests, and a cross-modal matching 

/ 

technique with a one week interval between test^ indicated the reliabil- 

.ity of the instrument. ' ^ ' ' 

I The data were analyzed to determine whether the scale possessed 

ra(tio characteriistics . Linear relationships were noted with regard 

the item geometric mean standard errors and<- increase in subjective 

etetima'tes, and the ratio of the geometri^^ and ai^ithme tic means and 

increase in subjective estimates. Evidence of consensus about the rank 

ordering of the scale itiems was found .with levels of significance beyond 

- " ■ . ' / ■ • 

1.001 being .obtained for Kendal's coefficients of concordance W for each 

•' " / _ 

[of the three groups. Analysis -of ^ovariance revealed age and years of 



iii 



schooling compieted to be significant sources of variance on several sea 
items and sex was found to have a significant effect on'^pnc item. 

The methodology followed- in the construction and administration 
of the scale were described .and suggestions were made for the implemen- 
tation of magnitude estimation scaling techniques^ iu adult res^rch. 
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THE CONSTRUCTION OF A MAGNITUDE ESTIMATION SCALE OF ADULT LEARNIN'G 



ADRIAN BLUNT 

Depaftment of Adult Education 
University of British Columb'ia 



INTRODUCTION ^ . . 

• Research goals in the emerging discipline of adu-lt education 

have been primarily concerned with the description and classification „ ' 

' [21 - 

of the field and related phenomena. Measurement has been relatively 

unsophisticated and data, has b*een collected mainly by nominal and ^ 
ordinal scales. The study of participation for example^has generally^ 
relied upon measures that do -not relate physical participation to learn- 
ing and which are not appropriate for sophisticated parametric analysis. 
The levels of measurement generally used, simply classify types of ,^ 
individuals as participants .or non-participants in one or more types 
of adult education. In rare instances when a more precise measure of 
participation is go^ght, indides which may possess the characteristics 
of an interval measure are developed by means of supposed experts who 
assess the '/edjucativeness'; of various types of participation. 

If adult education/research is to seek answers to fundamental 
questions and to investigatle the many complex intsr-relationships which 
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exist in the field, attention must be devoted towai^ds the two highest 
levels of measurement, interval and ratio scales. 

The purpose of this paper is to describe the 4Gvelopment of a 
ratio scale measuring subjective estimates of ^adult learning.- The scale 
was devel'&ped with the following criteria in'mind, that it: 

1. ' be appropriate for use in a wide range of settings, 

2. be administered' simply and quickly, 

3. produce a si-^gle score indicating the respondent ' s involve- 
ment in adult learning experiences, 

4. be appropriate for sophisticated parametric "analysis, • 

' 5. be capable of distinguishing between vario^us adult 

^ ■ . .. . ■ 

education activities by amount of expected learning, ffid 

6. contain a broad range of adult learning activities includ- 

Ing^t^) those likely to occur in the formal instructional 

and natural societal settings, (b) credit and non-credit 

activities^ (ijv) vocational , hobby and general interest 

learning activities. . 



SCALE TYPES AND CHARACTERISTICS _ ' 

Sciientific measurement involves the assignment of numbers to 
observations on the ba\sis of clearly defined rules. Differing means 

\ ■ ", . - . ; . \ 

of measurement have beeW, developed in accord with a variety of rules. 

*• ■ ■ • ■ . ' ' . ■ ' • 

. Stevens has identified four broad classes of measurement scales 
and has specified the mathematical assumptions which govern the 



permissibiei transformations on the scale numbers. The important differ- 
ence betj^^een Me four classes of scales is that, some are operr to more 
forms of mathemat^^al^,^tTeatments than are others, and consequently are 



more ^desirable in research 



Tfie four classes of senile, from the lowest levei of measurement 

to the highest, are nominal, ordinal, interval and ratio. Nominal, 

scales are used to categorize and HCila£>sify data^such as participants and 

non-participants, persis ters and drop-outs, .or resear.chers and instructors 

Ordinal scales enable observations to be ranked or ordered from most 

to least in terms of the attribute of interest, such as a list of names 

ordered by distance travelled to attend an adult education centre', or 

the completion of an assignment. Such measures would reveal who in a 

class travelled fi^^rthest to who travelled the least far, and who* in a 

group finished an assignment from the first person to finish, to the 

last to finish. No algebraic operations can be conducted with- either 

nominal or ordinal scale data. Neither can ranked data be meaningiully 

added or subtracted and only on monotonic transformations are ordinal 

'scales invariant. That is a transf oirmatipn which does not change the 

order of the observations is allowed, such as changing all twos to 

fives, threes to eights and fours to nines on a four ca^goty scale 

because no information is lost by such changes^ ^ 

With interval scaling it is possible to make statements not 

only about the rank order of observations , but also to state hov; far 

..apart the observations are on the scale. It is. possible therefore to , 
' ft ■ ? ■ • 

say that a person with an- IQ score of 110 is as far above the mean ' 

test score^of 100 as another individual with a score of x^O is beloW 
■ ' • . /' 



;the mean. The major difference betwe&n the interval and ratio scale ^ 
•is that the ratio 3cale has an- absolute rather than an arbitrary zero 
"jenabling observations such as' tiped performances to be taken, ' The most 
• important characteristic of* the 'ratio scale i"s that it remains invariant 

over all trans,f ormations when multiplied or divided by a constant. 

/ . . • ''^ ^ 

That is the rank order of points on the scale, the ratios ^f points on 

; the scale and the z^ero point are alli^ unaffected by division, or multipli- 

* '\ - ^ 

cation by a constant. Inte*rval scales are also invariant under linear 

transformations and are unaffected when a constant is added to each 

value after being multiplied or divided by another constant. 

Frequently in adult education research the fundamentalist' view 

of Stevens '''is ignored in the name of pragmatism and ordinal data is 

treated as though it had interval and ratio scale characteristics . 

Although there is support, in the literature for such action'-"^' ctie 

. breaking of assumptions underlying the theory of measurement ought not 

to be taken lightly. Although the question of the use of parametric 



I 



r and non-parametric tests in the literature is highly contentions there 
ought to be no questioning the wisdom of always striving to develop the 
highest level^ of measurement possible , Generally interval and ratio 
scaXe^^ are thought to be inescapably complex and tedious to construct, 
while nominal and ordihal scales are thought to be both easy to construct 
and acceptab]e by convention. For this reason alone many graduate 

students and reseatchers in the discipline do not devote** more j time to 

■ ' ' ' 

scale construction. • 

This study explores one means of constructing a ratio scale for 
ujge in an adult Education study by a relatively sSimple" psycho-physical 

, ^ •'5 

scaling procedure known magnitude estimation. 



• • • 

Magrlitude Estimation . ' ^ ' 

Originally developed by psychb-iiltysicists to measure perceived 
increasis'^in the magnitudes of such variables as loudness, brightness 
arid heaviness magnitude estimation has been found to have utility in the 
quantification of social norms,, opinions and attitudes- Examples of the 
successful application o f mag nitude estimation can be found in the^ 
diverse areas of aesthetic value of hanc^rlting , seriousness of the 
- offences of juvenile delinquents^ , the importance of Swedish monarchs 
occupational prestigr , and the magnitude of social re- ' djustments or 
.[5,6] 



li^fe^hangies 

The basis of the method rests on the human capability to match 
numbers to stimuli and stimuli to numbers in such a way as" to accurately 
estimate ratios between stimuli and also to adjust stimuli to match^ 
prescribed ratios. The generally used procedure in magnitude estimati^bh 
involves having the subject compare two stimuli presente<i simultaneously. 
One of the stimuli serves as a standard or referant and the subject 
•estimates the magnitude of the two,, or the ratio between 'the two stimuli. 
Subjects', responses can be obtained by the assigning of numbers, draw- 
ing lines of lengths proportional to the stimuli, squeezing hand grips, 
•drawing circles or" squares, increasing the magnitudei of another stimulus, 
or by. some other similar means; Stevens refers, to these procedures, 
as directmethods of scaling, and to the Thurstonian procedures as 
indirect, the distinction is that with indirect" methods respondents 
perform~a minimum of quantification. Usually they are asked which of 
•a pair of stimuli is the greater in terms of an observable characteristic. 
This data provides information to. rank order the stimuli, and later 



variability oE^udgements is- superimpos^e ranks in order to 

distribute subjects along fhe scale continuvim. With the ^direct method 

of magnitude estimation no psychometric assumptions are superimposed 

on the data at a later"date, The respondents petform the .quantification 

usually by assigning their own numerical value to the -response. On many 

attitudinal continua Thurstonian scales generated by indirect methods 

1iave "been foun'd to have an invariant' relation to the scale of magnitude 

developed by the direct Tirethods of magnitude estimation) 

« 

ih many experiments *over the' last .thirty ^years it has. b^en shown, 
from t^he subjective estimations of observers plotted against the magni- 
tudes of the stimuli observed j^^^ihat there is a greatj deal of agreement 



between ^individuals ' perceptions of magni 



tude. In. if act man may have 



an innate psychplogical capacity for makiig quantitative judgements 



about psycho-physical phenomena. It has 



been suggested that these 



' , / 

responses are' non-voluntary at the psychological level, i and it is ^lso 

possible as Shihn'''^^'' has speculated, that subjective responses to social 

stimuli may al^o be non-voluntary and that "the individual is in a 

sense a prisoner of his conditioned values/' If such is the case the 

Value placed on learning by an adult may be determined;>jby his previous 

experiences as a learner. As many adults have shared similar ..experi- 

ences in formal instructional settings both as pre~adults and adults 

it is conceivable^^that there is a social consensus i^egarding the magni- 

■ ' ( . . • f ' 

tude of learning that occurs in various activities* By quantifying 

^' . ' ^ - ■ 

these subjective estimates of^ learning it would be possible to assign 
"units of learning" to individuals participating in those activities 
indicating their exposure to or involvement in learning over a period 



of time.* If achieved, such measurement capability would allow research 

*r ' . \ . .. . 

into the extent of an individual's' commitmeiit , to learning and would be 

a major step beyond simply o}>serving acts of participation which 

do not take intQ.^eauTrf^elativ€ differences in expected learning.' 



INSTRUMENT DEVELOPMENT 



The Subjed^tive Estimation of Adult Learning Scale (SEALS) was 
developed over a three year period. |A total of 110 items for use in 
the scale were derived from adult edubatioii brochuriEiS j'" calendars , and 
circulars describing adult learning opportunities 'and from colleagues. 
The pool of items was used in several \:rial versions which were adminis- 
^tered to adult^educaticin stiJdents and participants in a wide range^of 
adult ei^cation activi/ties. Item analysi^es were conducted and tbose items 
with very large standard deviations indicting disagreement between 



raters were rejected. During fchis proi^ess items were edited to 

^ . \ ^ ■ 



clarity, the duration .of each of the activities was added to. t\\e. form, 
directions to the respondents were modified and the values of tne 



ensure 



"standardV* ^Item^were varied, ^ 

A modified form consisting of thirty-five items was te 
a sample of sixty vocation,al instructors and seventy-f ivci adult 



ted with. 



education graduate students and yielded satisfactory results for the - 
twenty~S3x items which now comprise the final form. The items include 
a range of credit, non-credit, vocati^n^, general'-interes t , in- 
s.titubional and self-directed Ifiarning activities, \ A, copy of the final 
form is at^tached. 
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Administration • ■ \ • ^ - 

— ■ . / ■ ■'/,,.■,■. ^ . . 

The SEALS was administered to a convenience sample of partici- 
pants in credit courses at the Vancouver C6nim.uriity College, students ,and 
faculty in the Department of Adult Education of the University of British: 

* Columbia and participants^ in the Vancouver School Bbard ajiult education 
program. Not all of the respondents^, fully completed the form and 
approximately 10 p^r cent of the college payticipants and 15 per cent. 

' of 'the school tfC^ai^d participants gave responses which xindicate£ that 

they did not und>,rstand the instrujCtion^ on the fio^rm. The incomplete 

and inaccurate responses w'ere rejected leaving 191 college respondents 

147 adult education 'Students\and faculty, ^and 165 schpol board resporidents 

, M \ ^ ^ 

for a total of 503 completed f'orms. j . , J 

At each administration of the form the respor>d^ents were given 

■ , - ■.• . / ' • ■ . 

an explanation of the intent of the* research and s;^veral examples of 

t' ' ' ■ * 

th(E rating procedure were presented .prior to the form, being distributed. 

' The ins true tio^is oil "t tic forni were read aloud and individuals, were en- 

y.couraged tp ask questions and seek guidance on. the procedure. There; 

was no discussion of thp actu^scale items or the.^values to b^ placed^ , 

on the items, andv there was no discussion ouce the groups began to 

A , \ 

ccmplete/the forms ..\ ;Ajbproximately half of the adult \education. "s tudents 
^responding had completed one or two of the developmental forms of the . 
SEALS and Were theref pre?\^skilled in this particular magnitude est/Lmation 
technique. ' 4;"^^ ' ^ - \. ». . ' 

^ In addition to responding to the 26 items on^ the form the 
respondents /were asked to provide information on their sex, number of 
'^ars' of schooling completed and their age. \ Of the total numbex:, pf 



, ■ ■ •/ , • ■ ■ 

respondents 62,6 per cent were male and 37.5 per cent were female; 
66.9 per cent had completed thirteen or more years of schooling, 26.6 
per cent had .completed niite 'to twelve years- and 6.5 per cent had com-^ 
pleted eight years or less of schooling.' There was no significant 
difference between males ' and • females 'by years of schooling completed 
(x = by sex and age. (x^ = 4. 05). A summary of the character- 

istics of thi respondents' by sex, years .of schooling and age is 
provided in ^ liable 1. . . ^ 



TABLE I 

CHARACTERISTICS OF RESPONDENTS BY SEX, AGE AND 
. YEARS; OF SCHOOLING CO>£PLEf ED ^ : 



Years of Schooling Completed 



Sex 



Age 



Male 



Female 



Total 



< 24 

25-34 

35-44 

> 45 



Total 



<8 Yrs 



No , % 



15 3.0 
U 3.0 



29 6.0 



0 0.0 

4. 0.8 

9 1.8 

16 '3.0 



29 6.0 



9-12 Yrs 



No % 



95 19 .\) 
42 8.0 



137 '27.0 




137 27.0 



> 13 Yrs 



No 



206 41.0 
131 26.0 



337 67.0 



185 37.0 

113 22.6 

26 -5 .,2 

^13 2.6 



337 67.0 



Totals 



NC^ 
— V- 



316 63.0 



187/ 37.0 



503 .100 .'0 



259 51.8 

149 29 . 8 

52 10.4 

. 43 8.6 



I 503 100.0 



r . ' ■ . ^ • . 

Results \ * ^ 

The mean item/'seores prodtfc^ed by the 503 -respandents are nre.sented 
In Table 2. As recommended by Steyens^ geometric means were calcu- 
lated to establish the mean item values. Arithmetic meanS|ar.e also 
p^^ented for purposes of comparison however as previous research has 
shown that the variability of magnitude estimations grows in proportion 
to the magnitude of the scores, geora^Jiric means are the most appropriate 
measure for ^ averaging over ' sub j ecfs . ^ 



or 



log 



■ logX^ + logX2 + log\ 



The relationsSip betwden- jthe geometric and arithmetic means can 
be clearly seen in Figure 1. Irregularities .in the downward descent of 
the arithmetic mean curve identify deviations from the} geometric mean 

. ■ ■ ■ ■ " ^ ■ 

ordering. ■ , ' 

The initial striking ^^feature of the .results is the agreement 
between th - three groups'^ in terms of the ranking of the items by their 
geometric means. Speiirman .rank order correlation coefficients were 
computed to test the relationship of the rankVorders be^tween the groups. 
The coefficients ^)b twined, all, significant b^.yond the"'. 01 level were 
collegp Vs scl^ocrrtoard r^ == 0.99, college vs adult education r.^ == 0.99, 
and school board vs adult education r =. 0.98. The correlation 
betwecn^lig^rSnk order of items as they ^appeared on the form af\d the 

■. 14' - • ■ . ' 



•TABLE <lJ^i^ ^ 11 

ARITWfETI^MEANS, GEOMETRIC MEANS AND GEO^^ETRIC MEAN RANKS BY INSTITUTIONAL 
AFFILIATION OF RESPONDENTS A:jD SCALE ITEMS 



Original 
Item 






Community College 


Adult Education 
Department 


School Board ^ 


Totals 
« 


Number 


Item 




, N 191 




N = 


1A7 


N « 165 


N - 503 








A 


\ 


Rank 


A 


G 


Rank 

\ 


A ^ 


G 


=lank 




G 


Rank 


11 


r 

Apt)renticeship 




1025 




/ 


929 


A 39 


1 


1831 


1119 


1 


1261 


625 


1 


20 


Vocational School Course 


ASA 


263 


2 


371 


267 


2 . 


*596 


390 


2 


A88 


301. 


2 


16 


University Credit Course 


29A 


219 


3 




226 


.3 


338 


282 


3 


300 


2A0 


3 


9 


University Credit Correspondence 


251 


182^ 


A 


230 


187 


A= 


261 


209- 


A 


2A8 


192 


A 


22 \ 


Community College Credit- 


201 


162 


. .5 


2li 


187 


A- 


211 


181 


5. 


'208 


176 


5 


2A 


High School Credit 




1A2 


125 . 


7 ■ 


16A' 


1A2 


7 


170 


;aa 


6 


158 


136 


6 


.. . 27 


Short Vocational Credit Course 


1A3 


1 

123 


8 


180 


152 


6 


151 


130 


8 


156 


133 


7 


3 


High School Credit 
Correspondence 




165 


126 


6 


I/O . 




0. 


166 


133 


7 


169 


132 


8 


13 • 


Non- credit College-" ' 




121 


100 


9= 




QA 




115 


.106 


9' 


llA 


101 


9 


1 


Standard Item ^ 




100 


r 

100 


9= 




1 nn 


1 1 


100 


100 


10 


100^ 


.100 


.10 -"-^ 


5 


Non-qredit Correspondence 


i31 


97 


11 






Q 

J 


121- 


' 90' 


11= 




99 


11 


- " 21 


Recreation Centre Lessons 


116 


93 


12 




1 n9 


1 n 


110 


90 


11= 


lis" 


95 


12 


•10 \ 


Programmed Text 




79 


62 


13 


111 


en 
ou 


1 J 


"73 


58 


13 


87 


63 


13 


17 


One Day Workshop 




63 


■ 50 


lA 




Al 




52 


Al 


16 


6f 


50 


1A = 


A 


Individual Lessons 




83 


55 


15 


/ o 


AA 


1 7 
i. / 


81 


A9 


lA 


SO , 

\ 


50 


1A = 


/ 15 


Read Non-fiction Book 




66 


51 


16 


/ ± 


/,Q 


1 c 
Lj 


59 


AA 


15 


1 

65 


A8 


. 16 


/ 23 


Labour Union Short Course 


65 


A.A 


17 




A ^ 

M J 


^ A 


A9 


35 


17 


58 


Al 


17 


U / 


Attended a One Day Convention 


58 


A2 


18 


55 ' 


34 


19 


50 


31 


18 


5A 


36 


18 


12 


Read" a News Magazine 




51 


35" 


19 


60 


A2 


. 18 


A8 


30 


19= 


53 


'35 


19 


7 


Guided Tour 




. 52 


3A 


20 


A9 


32 


21 


A8 


30 


19= 


50 


32 


^0 


18 


Educational TV Program 




A8 


35 


21 


A6 


33 


20 


35 


23 


21 


A3 


30 


21 


8 


Taped Lecture - - 




50 


31 


" 22 


39 


25 


23= 


36 


22 


22 


A2 


26 


22 


26 


Attended a' Public Show 




3.8 


26 


23= 


39 


26'^ 


22 


30 


. 19 


2A 


35 * 


2A 


23= 


25 


Attended a Public Lecture 


36' 


• 26 


23- 


. 3A 


25 


23-^ 


32 


*20 


23 - 


3A 


2A 


23- 


19 


Read a Daily Newspaper 




39 


26 


23- 


r 

- 37 


25 


23=« 


29 


16 


\ 


35 


22 




. 6 


Attended a Union or Professional- 
Meeting 


30 


20 


27 


31 


21 


. 26 


29/. 


17 


25- 


30 


19 


26 


2 


Listened to Kducational Radio 


35 


23 


26 


• ) 23 


i5 


27 


26 


17 


25f 


28 


18 


27 



total grbup item ranks was not significant (r = -0.047)-. 

Kendall's coefficient of concordance W was used to test'^the 

/■ 

extent of agreement between respondents ' rankings of the scale items 

tj • _ . 

within each of the three respqhdent groups. Unfortunately the computer 

program for Kendall's W was not capable of dealing witb the total number 

of respondents simultaneously^. The yalues 6f W obtained for each group 

indicated that the respondents assigned scores when used to rank the 

scale items resulted in consensus rankings. A high degree of agreement 

about' th^ ordering of the items generated W coefficients significant 

4iey^n4~the— rX)£)l level for each group. '^Community college W = 0.59 , 

adult education, department W 0.64, school board W = 0.66).. 



13 

«= . • , . 

The item receivings largest assigned units of l-^arning was 
apprenticeship. The geome--"''^' Tnean score assigned, to this item by the 
school board respondents (r.J"'0) was over twice as high as that 'assigned; 
to the item by .college respondents (495) and by adult education department 
respondents (439) / A similar pattern of responses was obtained for 
the six inonth vocational fc-'v:.- ving . course and the university credit' course 
.ranked second and third respectively.^ " However, the size of ^ the dif f er- 
•ence between the geometric group means was reduced from 100 per cent 
to 50 per cent and 25 per cent respectively with school "board respondents 
producing a geometric mean of 390 for the vocational course anc\ 282 for 
"the university cbyrse, the college respondents' preans were -263 and 219 
and those pi the adult education department 267 and 2-26 for the ^respective 
items V 

As was expected iteps of longer duration received .^larger scbre^^ 
th^n did items of short duration (r = 0.90, p < 0.001). Credit -courses i 
received the highest ratings following apprenticeship and vocational .. 
/ training, with university oc.edit having a geometric mean of 240 as 
compared to community college credit with 176 and high school ci-edit 
with 136 all for a period. of sixty hours duration. Study by. corres- 
pondence at the university"^a^iid high school credit levels were rated 
lower than co;rresponding regular credit activities even though one 7 
hundred hours of study time was allotted to them rather than sixty 
hours Cor the regular credit courses. A similar relationship held 
for' the non-credit correspondence item. the magnitude of this differ- . 
once between assigned units of learning was large for the, ^university ^ 
credit items with 240 as compared to 176 units, fcut relat,ively small 

17. 
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for. the high school credit with 136 to 132 units and for the non-^credit 
items 100 as. compared to 99 units of learning. 

Little difference was found between the ratings assigned general 
interest courses offered through a college(lOl) and a school district (100) 
and learning a hobby or recreational skill at aVecreation centre.(95) . 
These three items had the same duration of twenty hours. * The remaining 
items did not fall into any clear paftern of activities other than . • 
being structured individual and short duration formal activities includ- 
ing using a programmed text book(65) , taking individual lessons ( 50) , \ ^ 
attending a que day workshop(50) , tending a non-fiction bookC^/; , and 
'attending a stiort .course given by a union or professional association(41) . 
Finally, the remaining items .consis ted Of a94:ivities involving the mass 
.media with', news and cultural magazines ( 36) , educational television( 30) , 
reading a daily newspaper (22) , and listening to a serious radio pro- 
gram(18) being closely ranked" with '"public" events . such as . attending a 
• convehtion(35), taking a guided tour (32), attending a' public exhibitioner ^ 
show(2A) and attending a union or a professional association meeting(J9). 

; Although as previously mentioned course duration was correlated 
with' the magnitude of the item means (r = 0.90, p < .001^^, tha.s relation- 
ship is exp-^essed without regard to the ratio between the duration of 
'each activity and the assigned learning units. The^number of units ^ 
assigned per hour of learning activity, or the time-learning benefit 
ratio of eacli activity provides a different view of the data. Table 3 
shows the' items rank ordered by geometric mean number of: units of 
learning assigned per hour of learning activity. Tliose items of short- 
est duration had the highest assigned learning units per hour,, and 
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TABLE 3 



ITEMS RANKED BY TIME-LEARNING UNITS RATIO 



Rank 


/ Item 


-.,1 


G 


Duration 
(hours) 


— 

G/Duratipn 


— 

G Rank 


' 1 


Read News Magazine \ 




35 


1 


35 


19 


2 


. Educational TV Program\ 




'30 


1 


30 


21 


3 


Taped Lecture * \ 




26 


1 


26 


22 


4 


Read a Daily Newspaper 




22 




22 


25 


5 


Listened to Educational 


1 Radio 


18 


1 


18 


27 


• 6 


Guided Tour 




32 


2 


16 


20 ^ 


7 


Attended a Public Lectx 


Ire 


24 


2 


12 


23= \ 


8 


Labour Union Short Course 


41 


4 


10,2 


17 


9 


Union br Professional Meeting 


19 


2 


9,5 


26 


10 


One Day Workshop 




50 


6 


8,3 


14= 


11 


One Day Convention 




35 


6 


5,8 


19 


12= 


Individual Lessons 




50 


10 


5,0 


14= 


12= 


Standard Item 




100 


20 • 


5,0 . 


10 


12= 


Non-credit College 




101 


• .20 




9 


15= 


Read Non-fiction Book , 




48 


10 


4,8 


16 


15 ' 


Recreation Centre Lessons 


95 


20 


4,7 


12 


17- 


Attended a "Public Shov; 




24 


6 ; 


4,0 


.23= 


^ 17= 


University Credit Coursev . 


240 


20 ' 


4,0 ;: 


3 


19 


Short Vocational Credit /Course 


133 


. 40 


3,3 


7 


20 


Programmed^ Textboo]^ 


/■ 


65 


20 


" 3,2 


13 . : 


21 


Community "College Creda 


176 


. ^ 60 


2.9 


_/ 


22 


High School Credit / 




136 


60 


'■' 2.3 


o 


23 


University Credit-Corr/espon-' 












dence / 




-^\^92 


' 100 


1.9 


4 


24 


High School Credit-Corres- 






/ 






pondence 




132 


° i6o ' 


1. 32 


8 


25 


Non-credit Correspondence 


99 


100 


1.0 


•11 


26 


Vocational School Course - ^ 


301 


600 


0.5 


2 


"27 


Apprenticeship - 

i 




625 


7 


<0 . 5 


1 










(es timated) 






the original ranking of -ft ems by geometrie means i^ reversed. 

■ An adequate explanation of this finding cannot be provided at 

this time. Several hypotheses might be advanced including tl^e follow- 
ing: (1) The data confirms that immediacy of learning is highly valued 
by the adult and those activities which are intensive and of short 
duration are more highly rated learning experiences. (2) Adults see 
different learning activities in a qualitative sense, with immediate 
learning from the media and short term activities being- judged differ- 
ently' frQm learning acquired through long-term activities. (3) The 
results are an artificial product of the data, respondents did not take 
duration into account but merely thought of the potential for learning 
in each item. (4) The respondents' perceptions of the value of time 
increased as th^ units of measurement decreased, conversely their 
judgements of the value of time in learning decreased as the time 
intervals increased. (5) The ability to adequately assess learning 
may decrease as the duration and possible complexity of the learning 
activity increases. (6) A. combination of these factors, may be contri- 
butiug to the obtained results. ^- . . 

\ To test for the^effects of sex, years of schooling and age on ^ 
the estimates obtained for the total gr5up ai\^ analysis of c'ovariance 
was conducted using log estimates as the observations. A unbalanced 

2x3 factorial design with ag6 as the. covariate was used with a general 

\ [141 * . ' ' 

linear mod^l BMDlOV . The results of the analysis are summarised - 

in Table 4\ . . ' * . ^ 
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TABLE 4 
ANALYSIS OF COVARIANCE TABLE 



Source of Variance 


Items p < .05 


Items p < .01 • - 


1 


^ Sex 




11 " 


2 


Education 


7,. 15 


9,10,11,16,20,21,27 


3 


Sex: Education 




11 


4 


Covariate.-Age 

■ ' /' 

/ 


/ 4,14,16,19 

■ / - 

/ 


1,2,3,7,8,11,13, 
15,17-,18,20,25 ■ 





The covariate age was a significant source of variance on sixteen of the;^ 
' twenty-six' items tested, education was fo'und to be significant with nine 

/ ' ■ ■ ■ s 

items, and sex and a sex- education interaction were significant on one item, 

^ ■ ' ' ' / ■ ■ " 

Those items on which educatd/bn was found to be a significant influence 

■ ■ ■■.-.■/ . . _ " , ^ . ■ 

wore with the exception of^the taped' lecture and the progrtimmed textbook 

^ / ■ ■ ^- . ■■■ ■ ■ ■ • . • " 

concerned with formal iT)6tructional activities^] Respondents with more 

'years of schooling comt)ieted tended to assign greater estdniates of learning 

• ■ ' * ■ >A 

to ' the -activities than did the respondents with fewer years of schooling 

>■ 

completed. • . i. » 

Sex and an interaction between sex and education were significant 

. » . * r 

/ ■ » • t> ' . ■ 

sources of variance on the item, .attendance'' at a one day convention. 
Females asKigned higher units of learning to ^attending a convention than 
-did males. Also the higher the -number* bf years of schooling completed the • 
higher the assigned unitvS of learning by fema3<es, years of schooling 



completed did not influence male respondents. 

The subjects in the study ' were predominantly young, highly educated 
adults. Older respondents- in, the study tended to have completed fewer 
years of schooling than the younger respondents, and it is clear that if 
age had not been controlled in the study years of schooling would have . 
had greater effects upon the results. Consistently throughout the . 
SEAL^ younger respondents assigned higher units of learning to the items 

than did older respondents. . . 

■ * 

Reliability . ^ - . ' 

1' 

Scale reliability was estimated using two> methods test re-test and 
a cross-modal matching technique. Twenty-four graduate students in the 
department of adult; education'at the University of British Columbia 
completed the SEALS tv/ice, with a one month interval between the two 
administrations. The Pearson product moment correlation coefficients 
obtained between individual's test and re-test responses for each item 
ranged from 0.51 to- 0.68 and were signi-f^cant beyond the .01 level. A 
coefficient of 0.79 was obtained between the arithmetic mean scores of 
the til^enty-six items on the group's test re-tesit responses. 

S^eventeen doctoral students and faculty irieifibers of the adult 
education department responded to the SEALS in its. regular form and one 
week plater completed a second form which required lines to be drawn to 
indicate units of learning rather' *thau the assignment of numbers .. .The 
standard item was pre&cnted with a line ten centimetres long. The 
..order of the items rc'mained the &ame, and space was left on the form 



- X 
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i ■ ■ ■ ^ 

for lines, to be drawn beneath each item. Respondents were asked to 


/ 




draw the lines free hand. If the length of their proposed lines 






exceeded the width of the form, which was printed on 8 1/2 x 14 inch 

I ■ ' • 






4> paper, provision was made for additional lines to be drawn below the 






. item. The measurements of the lines were recorded in millimeffes. 






The maximum length of cbntinuous line possible- was 350 milimetres 






or 3.5 time's the length of the standard.^ Test re-test correlation 






coefficients for each ir * -anged from 0.49 to 0.60 and were signifi- ^ 






cant at the .05 level. A correlation coefficient of 0.66 significant at 


V, 




* > • 2_ t enis . 
the .01 level was obtained between the test retest arithmetic means of the 

The reliability of. the scale was considered to be acceptable 






with the obtained correlation coefficients. However, it must be..^ , . 
acknowledged . that the respondents did have pEior experience^ w4th the 

SEALS form in" its devdlopmenital^ stages- and^_they ,.wer^e^ 

^ ., . . ■ ~ '" .... 

magnitude estimat^ion technique.' The utility of alternate means of ' 
assigning unit scores to the items was successfully demonstrated*!:* 






the cross-modal matching technique. ^ ' 

• ' , ■ 

Ratio Properties ■ " 










Do the results obtained provide evidence that die SEALS actually 






cons tTtu tes^~a~"^atio scal^^ . * . " 






' * Although -the three groups of respondents studied do assign differ- _ . 






ent magnitudes to the .j.tcms, there is a large degree of ^agreement between 






respondents within the groups as to the rank, order, of the items, and the 
differences in magnitudes assigrjed between the groups are major only with 
-respect to three items. ^ ^ ' 
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A concensus regarding the subjective estimation of learning in tile 
activities described influenced by years of schooling completed and 
age does appear to exist. , ... , 

One major similarity between the groups can be sfeen in the distri- 
bution of the ratios of the geometric mean scores and the arithmetic 
mean scores for each item when plotted against the arithmetic mean item 

. /scores as shown in Figures 2-5., These distributions are almost identical 
in^ shape to that reported by Masuda and Holmes ^ ^ ^ in the development of 
'a ratio s.cale of life change events. The distributions indicate that 
as judgemental variance increases with ^the magnitude^ of the estimates 
it increases proportionally and linearly to the. increase in estimates. 
Such a relationship according to Masuda and Holmes'" supports the 

y general scientific law of relative variability and contributes to the 
validity of subjective magnitude estimation procedures being used in ^ 
psycho-social measurement. The tv/o liighes t ranking items apprentice- 
ship and vocational training have mean ratios V7hich deviate- from the 
ratios of other high ranking items. This observation suggests that the 
reliability of the two items in terms of their ratio scale properties 
may be open to doubt 1 

The relationship between the standai'd erfors of the geometric 
mean and the geometric mean item scores ds shown in Figure 6. This 
linear relationship (r = 0.98) supports the principle of measurement 



that variability in scored is a function ofv the magnit;ude of the scores/ 

•I ' / 

and as such is evidence of the scalable n^t,ure of the data. 
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Stevens reports that indirect scaling by^'pa^.red comparisons 
produces a scale which is a logarithmic ft^nction of the magnitude 
estimation scale. Since twenty-seven items create. 251 different paired 
items, this final step in the. examination of, the ratio properties of 
the SEALS presented major judging and computation difficulties. Future 
research is being planned to reduce the number of scale items ,and" to 



use an item sampling 



procedure to test the reXationshipxbetween direct 



and indirect scaling of the SEALS, 



CONCL.USION ^ 



sVc 



The devGlopment of* a ratio scale measuring subjective estimates 

of adult learning described^ in this paper demonstrates that the psycho- 

physical scaling technique of, magnitude estimation has utility fo.r the 

>«' • 

quantification of variables, in adult education. Although a definition ♦ 
of learning was not provided for the respondents, their responses based ^ 
on their own- und'erstanding of the construct indicates that a consensus 
regarding ''subjective estimates of learning may exist'. It is recognized 
that further research with the SEALS needs to be undertaken in ordei* to 
ascertain the variability of estimates contributed by respondtats ' 
personal experience, or lack of it^ as learnejs in each of the twenty-- 
seven scale activities and socio-economic characteristics including 
occupation, income, age and level of education. Application of the scale 

; . ■ J ' . 

in communities outside the Vancouver Metropolitan Area is also desirable. 
In addition a comparison of the results obtained from a Thurstonian 
paired comparisons procedure and the SEALS needs to be undertaken in 
the future to provide conclusive evidence of the exist^ce of ratio 
scale characteristics. ' - 

' V , ■ 

The applicatioji of magnitude GvlJtimatipn techniques to the quantifi 
cation of variables related to motivation; le^irning needs; attitudes 
towards adult eduGation methods and techniques ^.^eyaluation of program 

outcomes; setting priorities or goals for groups ; barriers to partici- 

' - . ■ ■ ■ • ' . ■« • • ■ 

.pation;-.,a^ tqaching-learning styles among others, may yield reliable and 

valid ratio scales of measurement where presently only ordinal and 
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questionable interval scales exist. Samples of.thcvsize used in this 
Study .may .not always be required ,^^nd frequently are not utilized by 
psycho-physicists. ^ In fact- one. of ^the areas where magnitude estimiation 

■ may be of greatest use in the discipline of adult education is in an ^ 
area^f relative^j^e^Xejot-^^ conduct 'of small experimental studies in 
Gontrc'llcd learning environments. Alternatives to- the traditional pencil 
and 'paper testing procedures exist with magnitude estimation and/with 
imagination researchers in the discipline may apply those alternatives 
to many quantification problem areas in :adult education research, 

Direct rather than Indirect scaling^ procedures appear .to offer 

. /many advantages to adult^ education reseaychers . Thei'r adaption would 
enable" researchers in .the -discipline to/^engage in more sophistioated 
data analyses without breaking the assumptio^is of the tests being 
, applied, and they should yield time-saving benefits as they are based on 
relatively simple procedures in both their development and their 
applies t ion, ' , - i' p 
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Figure 1 Geometric and Arithmetic Mean Item- Magnitudes. Ranked by 
Geometric Mean ^ ^ 
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RELATIONSHIP OF RATIO OF GEOMETRIC AMD ARITHMETIC MEANS AND ARITHMETIC 
MEAN ESTIMATES OF MAGNITUDE i 
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Figure 2 Community College Respondents n = 191 
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Figure -J Adult Hducntion Dcpartmcut Rc'.spondcnts n = 1A7 
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Figure 4 School Board Respondents n = 165 
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Subjective Estimation of Adults | earning Scale ^ 

We- are attempting to ^and out how much learning people think will 
occur- in various types of acti^^lties . Would you please judge each'of the 
twenty-six activities listed in the follWing two pages in terms of the 
amount of "leai ling that you would normally^xpect to^ occur 

As. a basAc unit for comparison we have described a standard for you 
to apply your judgements against. Think of attendance at a non-areditj 
sahooZ distvioiy adult educatiofi course such as automotive tune-up, wood7 
carving, pottery, French conversation or English improvement (10 weeks - 
2 hours per week) as resulting in 10,0 units of learning. Using the standard' 
described, assign the number of units of learnin2 you think will occur in 
each of the activities listed, - , 

The number of units, you choose to assign, may be considerably greater , 
or considerably less than 100. The number should be based upon the averdge 
amount of. learning that you would normally expect to occur in each activity. 
The number of units you assign could, range from 0 (no learning expected) to 
500 (five times more le^irning) or even higher. There ^are no uvper limits 
to the nunh^r of units you can assign. 

For example, if you think thajt^-thoroughly reading a daily newspaper 
is an activity that results in only one- fifth as much learning as thj standard^ 
that is the non-creditj school district adult education class you would assign 
one~f.ifth of the standard's 100 units which is 20 to reading the newspaper. 

Example ; - Units of 

^ ■ ' Learning 

Sta ndard .: Took a ndn-crcditj school dis tyH>ct^ adult 

education course such as automotive tune~up, 

wood carving, potter/, French conversation, 

or English improvement. (10 weeks - 2 hours 

per week) . . * 100 

Activity : Thoroughly read a Wjor daily newspaper. 20^ 

Please Turn Over 



34 



31 



- Units of 

Learning 

Standard ; Took .a non^credit^ schooL district adult education 
. course such as automotive tune-up, wood-carving, 
pottery, French conversation or English improvement 
(10 weeks - 2 hours per week). . . .f^ 100 

1. Listened to a serious or educationally valuable radio program 
such as "Cross Country' Check Up/I or "As It Happens." (1 Hour) 

2. Todk a high school credit course 'by correspondence. (100 . 

Hours) . 

3. Had indivi/dual lessons or tutoring to develop a recreational 
or hobby skill such as .tennis , swimming or music^^ lessons . . 

(10 Hours) ' ' . • 

4. - Completed a non-credit general interest course'of study ^by 

correspondence. (100 Hours) 

5. Attended* a labour Union, professional association or commun- . 

ity group meeting. (2 Hours) . 

6. Took a guided tour or educational visit to a-n art gallery, 
museum, factory, industrial plant or institution such as a * ^ 

, hospital or armed forces base. (2 Hours) 

7. ^ Listened-^ to a teaching tape or record on subjects such as . 

learning ''a foreign language, or improving one's job 

performance. (1 Hour) ^ ; ' 

8. Completed a university credit course by correspondence, . 

(100 Hclurs) . 

9. Worked through a programmed instruction text hook such as a - . 
"how to do it," or "teach yourself" book on car mech^iaics", 
bookkeeping or macrame. (20 Hours) , ^ 

10. Served an apprentice skip. (6 months full time at vocational 
school and two years indentured to an employer). 

11. Attended a one day convention or professional association 

annual' meeting. (6 Hours) ' • - ' ^ 

12. Took a coirTrrunity college non-credit course such as learning 
a foreign language, decorating your own home, basic 
carpentry, or English upgrading. (10 weeks - 2 Hours per 

week) ^ . ' 

13. Thoroughly read a news or cultural magazine such as Time, 
Newsweek, Reader's Digest or Macleans. (1 Hour) 



-'f Please Turn Over 

• - . 3 f). ■ / 

■ ■ ■ ■ ■ , . ■ / 
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Units of 
Learning 



Standard ; Took a non-- credit ^ school distvict adult education 
course such as aytomotive tune~up, wood-carving, 
pottery. French conversation, or English improve- 
ment. (10 weeks - 2 Hours per week)' 



100 



14. 
15. 
16. 
17. 

18. 
19. 

20. 

21; 
22. 
23. 

24. 

25. 
26. 



Read a serious, non- fiction book o'f your own, borrowed from 
a librarS', a friend, or from any other source-(10 Hours). 

Took a university credit course - other than by correspxxiadence . 
(20 weeks - 3 Hours per week) ' • 

Attended a one- day workshop or educational course on any 
subject. (6 Hours). 

Watched a serious, or eduodtionafLly valuable television // 
program such as the "National Geographic" series^. The "Accent , 
of Man," "Newsweek" or "W5". (1 Hour) 

Thoroughly read a major daily newspaper. (1 Hour) / 

Took a provincial vocational school training course. (6 months 
full time) 



i^tl( 



Toole a course at a recreation centre to learn a recreational 
or hobby skill such as tennis, skating, "'golf , bridg^,' . 
pottery or p'ainting. (10 weeks - 2 Hours per week) 

Took a college-levelj^ credit course at a community college ~ 
other than by correspondence. (20 weeks - 3 Hours per week) 

Attended a short t'rainino course offered by-^ labour union 
or a professional association. (2 weeks - 2 Hours per week) 

. Completcd\Q high school-level credit course giver)' by a 
school district adult education department . (20 weeks - 
3 Hours per week) 

Attended a public lecture - Not a political meeting. 
(2 Hours) ' 

Spent a iay at an automobile, agricultural, house or boat 
show. (6 Hours) 

Took a short, part-time vocational course for. credit toward 
a 'trade certificate, s^ch as elcctric-al cc/des, welding, 
typing, or air brake operation, (10 weeks - 4 Hours per 
week) 
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